Method for predicting the destination location of a vehicle

ABSTRACT

Aspects concern a method for predicting the destination location of a vehicle, comprising processing a local preference graph of a user of the vehicle having nodes corresponding to locations visited before by the user by a first graph neural network, processing one or more of a spatial graph representing information about geographical proximity of locations, a temporal graph representing information about locations which have been visited one after another by users and the time between the visits of the locations and a preference graph representing information about locations which have been visited one after another and the frequency of visits of the locations by a second graph neural network, combining the result of the processing by the first graph neural network and the result of the processing by the second graph neural network by at least one neural network layer and using the output of the at least one neural network layer as prediction of the destination location.

TECHNICAL FIELD

Various aspects of this disclosure relate to methods for predicting thedestination location of a vehicle.

BACKGROUND

Predicting the destination of a trip is a task in human mobility whichfinds several applications in real-world scenarios, from optimizing theefficiency of electronic dispatching systems to predicting and reducingtraffic jams. In particular, it is of interest in context of e-hailing,which, thanks to the advance of smartphone technology, has becomepopular globally and enables customers to hail taxis using theirsmartphones.

For predicting a user's next destination, models such as a deep learningmodel based on a neural network may be trained to predict the user'snext destination based on a user's visiting sequence. However, it isdesirable to increase the accuracy of such approaches and achieve anaccurate and reliable prediction of a vehicle's (or, equivalently, auser's) destination.

The publication “Graph Attention Networks” by Petar Velickovic et al.,2018, in International Conference on Learning Representations (ICLR) (inthe following denoted by reference 1) describes graph attention networks(GATs), which are neural network architectures that operate ongraph-structured data.

The publication “Semi-Supervised Classification with Graph ConvolutionalNetworks” by Thomas N. Kipf and Max Welling et al., 2017, inInternational Conference on Learning Representations (ICLR) (in thefollowing denoted by reference 2) describes graph convolutional networks(GCNs), which are neural network architectures that operate ongraph-structured data.

SUMMARY

Various embodiments concern a method for predicting the destinationlocation of a vehicle.

According to one embodiment, the method includes processing a localpreference graph of a user of the vehicle having nodes corresponding tolocations visited before by the user by a first graph neural network,processing one or more of a spatial graph representing information aboutgeographical proximity of locations, a temporal graph representinginformation about locations which have been visited one after another byusers and the time between the visits of the locations and a preferencegraph representing information about locations which have been visitedone after another by users and the frequency of visits of the locationsby a second graph neural network, combining the result of the processingby the first graph neural network and the result of the processing bythe second graph neural network by at least one neural network layer andusing the output of the at least one neural network layer as predictionof the destination location.

According to one embodiment the method further includes processing auser-user graph representing information about similarity of users interms of the locations they have visited by a third graph neural networkand combining the results of the processing by the third graph neuralnetwork with the result of the processing of the first graph neuralnetwork and the result of the processing of the third graph neuralnetwork by the at least one neural network layer.

According to one embodiment the spatial graph has nodes corresponding tolocations, edges between nodes if the locations corresponding to thenodes are geographically near each other and edge weights depending onthe geographical proximity of the locations corresponding to the nodesconnected by the edges.

According to one embodiment the temporal graph has nodes correspondingto locations, edges between nodes if the locations corresponding to thenodes have been visited one after another based on the visits'timestamps and edge weights depending on the time between the visits ofthe locations corresponding to the nodes connected by the edges.

According to one embodiment the preference graph has nodes correspondingto locations edges between nodes if the locations corresponding to thenodes have been visited one after another in all users' historicalsequential visits and edge weights depending on the frequency with whichthe locations corresponding to the nodes connected by the edges havebeen visited one after another.

According to one embodiment the user-user graph has nodes correspondingto users and edges between nodes if the similarity of the userscorresponding to the nodes in terms of the locations they have visitedis above a predetermined threshold.

According to one embodiment processing at least one of the spatialgraph, the temporal graph and the preference graph includes selecting asub-graph of the respective graph and feeding the sub-graph to thesecond graph neural network.

According to one embodiment selecting the sub-graph of a graph includesselecting nodes of the graph by one or more random walks through thegraph.

According to one embodiment the one or more random walks depend on theedge weights of the graph.

According to one embodiment a multiplicity of random walks are performedon the graph and nodes which have been visited most in the random walksare selected for the sub-graph.

According to one embodiment processing at least one of the spatialgraph, the temporal graph and the preference graph includes selecting aplurality of sub-graphs of the respective graph and feeding thesub-graphs to different sub-graph neural networks of the second graphneural network.

According to one embodiment selecting a plurality of sub-graphs for agraph includes selecting at least one sub-graph having nodes adjacent toone or more nodes of the vehicle's user's historical destination visitsequence and selecting at least one graph selected by at least onerandom walk through the graph.

According to one embodiment the method includes processing at least twoof the spatial graph, the temporal graph and the preference graph byfeeding, for each of the at least two graphs, a first sub-graph havingnodes adjacent to one or more nodes of the vehicle's user's historicaldestination visit sequence to a first set of sub-graph neural networks,and a second sub-graph having nodes selected by one or more random walksto a second set of sub-graph neural networks, mean pooling the result ofthe first set of sub-graph neural networks, mean pooling the results ofthe second set of sub-graph neural networks and combining the results ofthe mean poolings by the one or more neural network layers.

According to one embodiment the at least one neural network layerincludes at least one of a dropout layer and a linear layer.

According to one embodiment the method includes processing all of thespatial graph, the temporal graph and the preference graph by the secondgraph neural network.

According to one embodiment at least one of nodes corresponding tolocations have trainable features, nodes corresponding to users havetrainable features, the graph neural networks have trainable weights andthe at least one neural network layer has trainable weights.

According to one embodiment the method includes setting one or more ofthe trainable features and the trainable weights by a training procedureusing historical trips of the users.

According to one embodiment the method further includes selecting avehicle for making a trip with the user depending on the predicteddestination location from a plurality of candidate vehicles.

According to one embodiment the method includes predicting a traveldistance from the prediction of the destination location and selectingthe vehicle from the plurality of candidate vehicles depending on thepredicted travel distance.

According to one embodiment the method includes selecting the vehiclefrom the plurality of candidate vehicles such that the selected vehiclehas sufficient fuel or battery to travel the predicted travel distance.

According to various embodiments, a server computer is providedincluding a radio interface, a memory interface and a processing unitconfigured to perform the method of any one of the above embodiments.

According to one embodiment a computer program element is providedincluding program instructions, which, when executed by one or moreprocessors, cause the one or more processors to perform the method ofany one of the above embodiments.

According to one embodiment a computer-readable medium is providedincluding program instructions, which, when executed by one or moreprocessors, cause the one or more processors to perform the method ofany one of the above embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood with reference to the detaileddescription when considered in conjunction with the non-limitingexamples and the accompanying drawings, in which:

FIG. 1 shows a smartphone in communication with a server.

FIG. 2 illustrates the processing of a GAT (Graph Attention Network)layer for a graph node as an example of a GNN (Graph Neural Network)layer.

FIG. 3 shows an STP (spatial, temporal, preference)-GNN according to anembodiment.

FIG. 4 illustrates a case where random walk masked self-attention mayimprove the prediction achieved by using adjacency maskedself-attention.

FIG. 5 shows an STP-UGNN (user GNN) according to an embodiment.

FIG. 6 shows an attention plot for one out of the eight GNN layers fornewly explored POIs (points of interest) a user has never visitedbefore.

FIG. 7 show an attention plot for POI-POI attention and an attentionplot for user-user attention.

FIG. 8 shows a flow diagram illustrating a method for predicting thedestination location of a vehicle according to an embodiment.

FIG. 9 shows a server computer according to an embodiment.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawingsthat show, by way of illustration, specific details and embodiments inwhich the disclosure may be practiced. These embodiments are describedin sufficient detail to enable those skilled in the art to practice thedisclosure. Other embodiments may be utilized and structural, andlogical changes may be made without departing from the scope of thedisclosure. The various embodiments are not necessarily mutuallyexclusive, as some embodiments can be combined with one or more otherembodiments to form new embodiments.

Embodiments described in the context of one of the devices or methodsare analogously valid for the other devices or methods. Similarly,embodiments described in the context of a device are analogously validfor a vehicle or a method, and vice-versa.

Features that are described in the context of an embodiment maycorrespondingly be applicable to the same or similar features in theother embodiments. Features that are described in the context of anembodiment may correspondingly be applicable to the other embodiments,even if not explicitly described in these other embodiments.Furthermore, additions and/or combinations and/or alternatives asdescribed for a feature in the context of an embodiment maycorrespondingly be applicable to the same or similar feature in theother embodiments.

In the context of various embodiments, the articles “a”, “an” and “the”as used with regard to a feature or element include a reference to oneor more of the features or elements.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items.

In the following, embodiments will be described in detail.

An e-hailing app, typically used on a smartphone, allows its user tohail a taxi (or also a private driver) through his or her smartphone fora trip.

FIG. 1 shows a smartphone 100.

The smartphone 100 has a screen showing the graphical user interface(GUI) of an e-hailing app that the smartphone's user has previouslyinstalled on his smartphone and has opened (i.e. started) to e-hail aride (taxi or private driver).

The GUI 101 includes a map 102 of the user's vicinity (which the app maydetermine based on a location service, e.g. a GPS-based locationservice). Further, the GUI 101 includes a field for point of departure103 (which may be set to the user's present location obtained fromlocation service) and a field for destination 104 which the user maytouch to enter a destination (e.g. opening a list of possibledestinations). There may also be a menu (not shown) allowing the user toselect various options, e.g. how to pay (cash, credit card, creditbalance of the e-hailing service). When the user has selected adestination and made any necessary option selections, he or she maytouch a “find car” button 105 to initiate searching of a suitable car.

For this, the e-hailing app communicates with a server 106 of thee-hailing service via a radio connection. The server 106 includes adatabase 107 knowing the current location of registered drivers, whenthey are expected to be free, has information about traffic jams etc.From this information, a processor 110 of the server 106 determines themost suitable driver (if available) and provides a estimate of the timewhen the driver will be there to pick up the user, a price of the rideand how long it will take to get to the destination. The servercommunicates this back to the smartphone 100 and the smartphone 100displays this information on the GUI 101. The user may then accept (i.e.book) by touching a corresponding button.

As mentioned above, when the user touches the destination field 104, theGUI 101 may display a list of possible destinations of the users' trip.According to various embodiments, this may include a list of predicteddestinations.

This means that as the user opens the e-hailing application, the server106 may predict the next POI or drop-off point the user intends to go(i.e. the destination of his or her trip) based on trip data (ofprevious trips) 108 which the server has in its database about this andother users. It should be noted that the database 107 may be distributedover various servers (e.g. be maintained by a cloud) which the server106 may contact via a communication network.

The server 106 may have a memory 109 storing a trained destinationprediction model 111 which the processor 110 may run to predict one ormore next destinations of the user (e.g. a list of 10 most probabledestinations of the user) from the user information 108.

For example, given the user's user ID and a list of historically POIs(drop-off points, i.e. destinations) of the user, which may be includedin the user information for that user in the data base 107, theprocessor 110 may, by running the model 111, predict a ranked list ofall the POIs with its respective probabilities on the likelihood of theuser visiting that POI next. This means the model 111 scores all POIs ina region around the user to recommend POIs that the user may or may nothave visited before, but could still be visiting in the future, makingit a challenging task for the model 111 to learn. The processor 108 maythen choose the highest ranked POI from the ranked list as the finaldestination recommendation/prediction and communicate it to the users'smartphone if only the best recommendation should be displayed to theuser on his or her smartphone. Alternatively, the server 106 cansimilarly recommend the top 5 or 10 etc. POIs by extracting from thesame ranked list accordingly.

According to various embodiments, the model 111 (and the correspondingmethod to predict or recommend POIs) is based onSpatial-Temporal-Preference (STP) factors. For example, it is assumed inthe model 111 that a user is keen to visit a POI next because it isnearby (spatial), or may be because he or she would like to visit a cafePOI next after having lunch at a restaurant POI (temporal). Also, if theuser has a preference of mostly visiting shopping malls, then the model111 may be constructed or trained such that it is assumed in the model111 that other shopping malls visited by other users could be helpful(preference). In view of these STP factors, in the following, threeexemplary models (which may be used as model 111) are described withincremental improvements in terms of achievable prediction accuracy:

-   -   1) Local Preference GNN (LP-GNN)—This can be seen as a basic        model that exploits user's local preferences but does not        consider the STP factors mentioned and the visit patterns of        other users.    -   2) Spatial-Temporal-Preference GNN (STP-GNN)—This can be seen as        explore-exploit model which exploits users' local preferences        (as done in LP-GNN) and explores new POIs which the user has        never visited before (but could visit in future) from STP graphs        constructed using both the adjacency and random walk masked        self-attention options. By learning the explore-exploit        tradeoffs during training of the model, this ensures that the        user's local preferences are not ignored entirely.    -   3) Spatial-Temporal-Preference User GNN (STP-UGNN)—This can be        seen as the most complex model of the three examples. It        includes a User GNN to allow using information for a user from        other similar users.

All three of the above exemplary models are implemented in form of aneural network including at least one Graph Neural Network (GNN). AGraph Neural Network is a sequence of Graph Neural Network Layers (alsoreferred to as GNN layers in the following).

A GNN layer may for example be a GAT (Graph Attention Network) layer.The input of a GAT layer is a set of node feature vectors h={{rightarrow over (h)}₁, {right arrow over (h)}₂, . . . , {right arrow over(h)}_(N)} where N is the number of nodes of the graph processed by theGAT layer. The layer produces a new set of node features (of potentiallydifferent cardinality) as its output for each node. In order to obtainsufficient expressive power to transform the input features intohigher-level features, the GAT layer includes a shared lineartransformation which is parameterized by a weight matrix and applied toevery node. Further, a shared attentional mechanism computes attentioncoefficients α_(ij) that indicate the importance of node j's features tonode i.

FIG. 2 illustrates the processing of a GAT layer for one node 201 as anexample of a GNN layer.

The input features for the node 201 are given by an input feature vector{right arrow over (h)}₁. The output features for the node 201 are givenby an output feature vector {right arrow over (h)}′₁. The attentioncoefficients α_(ij) define the impact of the feature vectors {rightarrow over (h)}₁, {right arrow over (h)}₂, . . . , {right arrow over(h)}₆ of neighbouring nodes 202 in the graph on the output featurevector {right arrow over (h)}′₁.

The output feature vector of the ith node is for example calculatedaccording to

${\overset{\rightarrow}{h}}_{i}^{\prime} = {\sigma\left( {\sum\limits_{j \in \mathcal{N}_{i}}{\alpha_{ij}W{\overset{\rightarrow}{h}}_{j}}} \right)}$

where

is some neighbourhood of the ith node in the graph and a is a nonlinearfunction such as ReLu. GAT layers and GATs as they may be used accordingto various embodiments are described in more detail in reference [1].

Alternatively or in addition, for example, Graph Convolutional Networklayers may be used as described in more detail in reference [2]. One keydifference of GCN and GAT is that GAT learns weights of adjacent nodeswhereas GCN does not.

LP-GNN

Input:

-   -   Previous POI (e.g. POI where the user is currently, i.e. has        gone to last)    -   Local Preference Graph        Output: Ranked list of all POIs for recommendation

For an LP-GNN model, a local preference graph is constructed for eachuser, with each node representing a POI that the user has visited beforeand all nodes are fully connected because it is a complete graph, builtfrom historical data. This graph serves to encapsulate the individualuser's POI-POI relationships and semantics.

Thus, according to one embodiment, a Local Preference Graph is anundirected complete POI-POI graph for each user u_(m), denoted as u_(m)^(G)=(V_(u) _(m′) , E_(u) _(m) ) where V_(u) _(m) and E_(u) _(m) aresets of POIs (coming from a training set) and unweighted edges,respectively. All pairs of POI vertices are connected, forming acomplete graph.

Given a previous POI to predict the next POI for the user, the basicidea of a LP-GNN neural network, e.g. used as model 111 and one or moreGNN layers, is to construct a numeric vector or a representation basedon the neighbouring POIs of the previous POI from the local preferencegraph through weighted average, where the weights are learnt when themodel is trained. This essentially allows the model to refer to othersimilar POIs in the local preference graph when computing arepresentation for the previous POI. Intuitively, the local preferencegraph serves as a form of domain knowledge to help the model compute arepresentation of the previous POI. With the representation as a numericvector, the model then predicts a ranked list of POIs where the next POIactually visited should be highly ranked after training. For example,when a city has 10,000 POIs, the numeric vector specifies a probabilitydistribution of all 10,000 values by softmax values (i.e. the vector hasa dimension of 10,000. Sorting the components of the vector, e.g. in thedescending order, gives the top K (e.g. the top 5 or 10) POIs forprediction or recommendation.

It should be noted that the LP-GNN model only considers the userspecific local preference and does not consider STP factors based on thevisiting patterns of other users. This may be sub-optimal.

STP-GNN

Input:

-   -   Previous POI    -   Graphs (local preference, spatial, temporal and preference)        Output: Ranked list of all POIs for recommendation

FIG. 3 shows an STP-GNN 300 according to an embodiment.

STP-GNN can be seen as an explore-exploit model that balances theexploitation of local preference factor, as done in LP-GNN, andexploration of new POIs which the user has never visited before viaglobal STP factors, supported by the visiting patterns of other users.For example, if the user has a niche shopping mall local preference inhow he/she visits POIs, this is well captured with LP-GNN and isimportant. However, he/she may visit new nearby (spatial) shoppingmalls, or new shopping malls tend to be visited closely in time by otherusers (temporal), or may be other new shopping malls visited by otherusers who like shopping malls (preference). Hence, the STP factors canhelp learn the relationships among POIs to support the recommendationtask.

However, both local preference and global STP factors are incorporatedbecause if the model only focuses on using information from newcandidate POIs that the user has not visited, but might or might notvisit in the future, the model would not be able to use (and learnduring training) the local preference of the user, of which it can beassumed for certain that it is representative of the preference of theuser. Similarly, if the model just uses the local preference factor, asdone in LP-GNN, then it would not bother to explore new candidate POIsthat could be spatially, temporally or preferentially similar. In bothscenarios, they are sub-optimal. Hence, the STP-GNN according to variousembodiments balances the exploitation of local preference andexploration of global STP factors.

For this, the following STP graphs are derived (e.g. by processor 110)from the local preference graph 313 for implementing a STP-GNN model(e.g. as model 111):

-   -   1) Spatial Graph 307: Nodes of POIs and they are connected if        they are within the top 10 nearest POI based on the distance of        their locations. This would essentially connect nearby POIs on a        graph.        -   Thus, according to one embodiment, a Spatial Graph is an            undirected POI-POI graph G_(s)=(V_(s), E_(s)) where V_(s)            and E_(s) are sets of POIs and edges, respectively. A POI            node v_(i) has adjacency (i.e. an edge) to a POI node v_(j)            if the POI of v_(j) is within the top σ (e.g. σ=10) nearest            POIs to the POI of v_(i) based on the Euclidean distance            Δd=d(v_(i), v_(j)). The edge weight between an adjacent pair            is

$\frac{1}{\Delta d}.$

-   -   2) Temporal Graph 308: Nodes of POIs and each pair of POIs are        connected if they are visited next based on timestamps,        regardless of the user. This aims to capture POIs that tend to        be temporally related, such as going to a cafe after lunch at a        restaurant. Thus, according to one embodiment, a Temporal Graph        is an undirected POI-POI graph G_(t)=(V_(t), E_(t)) where V_(t)        and E_(t) are sets of POIs (coming from the training set) and        edges, respectively. A POI node v_(i) has adjacency (i.e. an        edge) to a POI node v_(j) if the POI of v_(j) has been a next        visit from v_(i) (in the training set). The edge between an        adjacent pair is

$\frac{1}{\Delta\hat{t}}$

-   -    where Δ{circumflex over (t)} is the average time interval        between the visit of v_(i) and the visit of v_(j).    -   3) Preference Graph 309: Nodes of POIs and they are connected if        they have been visited sequentially before by any user. This        seeks to learn about the unique preferences of users e.g.        shopping mall preference.        -   Thus, according to one embodiment, a Preference Graph is an            undirected POI-POI graph G_(p)=(V_(p), E_(p)) where V_(p)            and E_(p) are sets of POIs (coming from the training set)            and edges, respectively. A POI node v_(i) has adjacency            (i.e. an edge) to a POI node v_(j) if the POI of v_(j) has            been a next visit from v_(i) (in the training set). The edge            between an adjacent pair is freq(v_(i), v_(j)) where freq is            the count function of POI pair occurrences.

Given the constructed STP graphs 307 to 309, according to oneembodiment, the STP-GNN model includes six GNN layers 301 to 306 tocompute a numeric vector or representation of an exploration phase,allocating three (301 to 303) to adjacency masked self-attention andthree (304 to 306) to random walk masked self-attention. According toadjacency masked self-attention, a feature vector of a node has onlyimpact on the output feature vector of another node if there is an edgebetween the two nodes.

According to random walk masked self-attention is used to attendhigher-order neighbours on the STP graphs (i.e. nodes which are furtherapart than one edge) due to the drawbacks of the adjacency maskedself-attention option in certain cases.

FIG. 4 illustrates a case where random walk masked self-attention mayimprove the prediction achieved by using adjacency maskedself-attention.

In the example shown, a shopping mall POI 401 is connected to othershopping mall POIs 402. Thus, by using only adjacency maskedself-attention, only the feature of the shopping mall POIs 402 (i.e.first-order nodes) have an impact on the output feature vector or POI401.

However, POIs that are not directly connected but nearby on the graph(i.e. higher-order nodes), such as a metro nodes 403, can help to betterpredict the next POI which is for example a metro 404. Therefore,according to various embodiments, random walk masked self-attention isused to consider nodes further away on the graph but yet relevant.

It should be noted that according to various embodiments, the six GNNlayers 301 to 306 are not provided with a complete spatial graph 307,complete temporal graph 308 and complete preference graph 309 but areprovided, by a POI embedding 314 (which can be seen as an input layer)with the following:

-   -   the three GNN layers 301 to 303 allocated to the adjacency        masked self-attention option are supplied as input with a        subgraph (of the spatial graph 307, temporal graph 308 and        preference graph 309, respectively) including the considered POI        node (e.g. the current POI of the user, i.e. the “previous” POI)        and adjacent nodes (all adjacent nodes or at least those, e.g.        10, which are connected to the POI node with edges having the        highest edge weights)    -   the three GNN layers 304 to 306 allocated to the random walked        masked self-attention option are supplied as input with a        subgraph including the considered POI node (e.g. the current POI        of the user, i.e. the “previous” POI) and nodes found by random        walks through the spatial graph 307, temporal graph 308 and        preference graph 309, respectively.

For example, for each of the graphs (i.e. spatial 307, temporal 308 andpreference 309), the POI embedding 314 uses random walks to sample nodesin the respective graph 307 to 309 as input to the GNN layers 301 to306. An example of the sampling process (e.g. for the spatial graph 307)is as follows:

-   -   a) Perform 1,000 random walks on the graph which leads to 1,000        lists of nodes. The number 1,000 is an example and may be        calculated as the number of nodes in the respective graph (e.g.        spatial graph 307) times the number of random walks to be        started from each node.    -   b) Filter the 1,000 lists of nodes such that only lists are kept        that contain at least one POI historically visited by the user        (e.g. the 1,000 lists are filtered to 400 lists).    -   c) Perform a frequency ranking of the filtered lists to identify        the top 10 POIs which are used as input to the respective GNN        layer (e.g. GNN layer 304). These top 10 POIs would be the final        sampled POIs.

This random walk-based generation of a subgraph is for example also donefor the temporal graph 308 and the preference 309 such that the GNNlayers 304, 305, 306 each have their own top 10 POIs generated fromrandom walks. Thus, each of the six GNN layers 301 to 306 for examplehas a different set of 10 POIs as input.

According to one embodiment, as indicated above, the STP graphs 307 to309 have edge weights. These may be used by the POI embedding 314 tobias the random walks. For example, in the spatial graph 307, randomwalks are biased to nearby POIs rather than POIs which are far away.This means that while performing the random walk, there areprobabilities to guide the random walk which are for example based onthe normalized edge weights of each graph. For example, given a node(POI) A, it has to be decided whether to traverse next to either node(POI) B or C. The edge weights

$\left( {e.g.\frac{1}{\Delta d}} \right.$

for the spatial graph, wnere Δd is the geographical distance between thePOIs) of the pairs A→B and A→C are normalized into probabilities, suchthat it would be in 0 to 1 interval and the higher probability will bemore likely chosen to perform the walk (e.g. A→C has higher probabilityas it has higher edge weight because they are nearer). Hence, this isdependent on the edge weight and is different for each STP graph as theedge weight definition is different for the STP graphs.

The outputs of the GNN layers 301 to 303 allocated to the adjacencymasked self-attention option are combined via a first mean pooling 310and the outputs of the three GNN layers 304 to 306 allocated to therandom walked masked self-attention option are combined by a second meanpooling 311.

The results of the mean poolings 310, 311 are combined by a first linearlayer 312.

In addition to the GNN layers 301 to 306 operating on STP (sub-)graphs,the STP-GNN 300 includes an LP-GNN 313 as described above.

The output of the LP-GNN 313 is combined with the output of the firstlinear layer 312 by a second linear layer 316.

Thus, a single STP representation built from the six GNN layers 301 to306 that represent the exploration, is combined by the linear layer 316with the LP-GNN result which represents the exploitation. The trainingof the second linear layer 316 allows learning weights that balances theexplore-exploit trade-offs.

The result of the second linear layer 316 is processed by a dropoutlayer 317 and the remaining values are fed to a third linear layer 318which produces a ranked set of POI predictions.

The dropout layer 317 to ignore certain parts of its input vector. Forexample, the second linear layer outputs a vector (e.g. [0.4, 0.1, 0.6,0.3]) and the dropout layer deactivates certain part of the vector inorder to force the model 300 to rely on the remaining part of the vectorand yet still achieve good prediction performance. For the example, witha dropout probability of 0.5, the vector could become [0, 0.1, 0, 0.3],where the first and third components are “deactivated” or intentionallyturned to zeros. In the next forward run of the overall model, otherparts of the vectors are likely being deactivated (based on the dropoutprobability).

STP-UGNN

Input:

-   -   Previous POI    -   Graphs (local preference, spatial, temporal, preference, user)        Output: Ranked list of all POIs for recommendation

FIG. 5 shows an STP-UGNN 500 according to an embodiment.

The STP-UGNN 500 can be seen as an extension of STP-GNN of FIG. 3 .Accordingly, it includes components/operates on graphs 501 to 518 as theSTP-GNN 300.

Including a numeric vector to represent the respective users whenpredicting the next POI is effective but may overfit the model to giveonly high probabilities to POIs the user has been before, but not otherPOIs. Therefore, the STP-UGNN 500 further includes a User GNN (UGNN) 519with the goal of allowing the model to incorporate information (andlearn) for a user from other similar users. For this the processor 110,for example, constructs a User Graph 520, where all nodes correspond tousers and the nodes are connected if they have some similarity in theirpast POI visit sequence. The similarity of a first user with pastvisited POI set A to a second user with past visited POI set B is forexample based on the Jaccard Similarity Coefficient

${J\left( {A,B} \right)} = \frac{❘{A\bigcap B}❘}{❘{A\bigcup B}❘}$

of their historically visited POIs. For example, two nodes are connectedif they have a similarity above 0.2. The variables used for thesimilarity computation are just historical POIs for the users in thetraining set.

Thus, according to one embodiment, a User Graph is an undirecteduser-user graph G_(user)=(V_(user), E_(user)) where V_(user) andE_(user) are sets of users (e.g. of all users) and edges, respectively.A user node v_(i) has adjacency (i.e. an edge) to a user node v_(j) ifthere Jaccard similarity coefficient is above, for example, 0.2.

Similar to how LP-GNN 200 and STP-GNN 300 apply GNN layers to learnPOI-POI relationships, the UGNN 519 includes a GNN layer 522 to computea representation for the user based on himself and his similar users, toincorporate (and learn during training) user-user relationships from theUser Graph 520 provided by a user embedding layer 521 (which may be seenas an input layer).

The output of the UGNN's GNN layer 522 is fed (together with the outputof the second linear layer 516) to the dropout layer 517 whose output isfed to the third linear layer 518 which produces the ranked set of POIpredictions as described with reference to FIG. 3 .

The UGNN′ GNN layer 522 takes an input of a tuple of a nodecorresponding to a user and connected nodes in the user graph 520, i.e.nodes corresponding to similar users. Given the tuple, the GNN layer 522computes a weighted sum (where the weights are trainable) of the user'sfeature vector to its similar users' feature vectors. This weighted sumis the output of the GNN layer 522. Thus, the weights of UGNN 519 can beseen to represent how much the model should extract from similar user'sfeature vectors for a user in order to perform well on theclassification task.

It should be noted that the weights of all linear layers 312, 512, 316,516, 318, 518 as well as the weights of the GNN layers 301 to 306, 501to 506, 512 are trainable.

Furthermore, each graph node, whether corresponding to a POI or to auser, is a vector of weights that is learnt and tuned when the model istrained.

This means that the models 300, 500 may be trained using training dataincluding multiple training data sets (each having a previous POI and aPOI visited after the previous POI, i.e. a sequence of historicaldestination POIs, e.g. from an e-hailing service, such as previous tripdata 108) to correctly predict the POI visited after a previous POI.After the training, the model may be used (e.g. by processor 110) topredict the POI that a user wants to go to (or that should berecommended to the user) from a previous POI (i.e. a POI that the userwent to last or is currently at).

According to one embodiment, the next destination is predicted, based onthe vehicle user's sequence of only all past destination points.According to one embodiment, all GNNs find adjacent nodes differently,e.g. LP-GNN uses only the previous destination location, STP-GNN usesnot just the previous destination location, but the whole user'shistorical set of destination POIs and UGNN uses the user node itself tofind adjacent similar users. In particular, according to one embodiment,for only STP graphs, the user's historical destination visit sequencethat has several POIs is used, and respective sub-graphs are foundindividually, then a union of all the sub-graphs is taken to form asingle sub-graph to be used. The idea is to rely on the whole historicalsequence rather than just the previous POI to find STP POIs that canhelp the task. Hence, in case of STP graphs, not just the previousdestination location, is used but all of the historical POIs for therespective user.

In particular, the user features of a node corresponding to a user ofthe user graph 520 is a vector of learnable weights to represent theuser, which will be optimized during training such that the modelperforms well for the prediction task.

It should be noted that training data may be pre-processed to ensurerobustness, e.g. POIs may only be kept if they have been visited by acertain amount of users (e.g. 10). Users may be kept even if they havelittle visit counts (e.g. fewer than 10).

It should further be noted that each of STP-UGNN's eight GNN layers 501to 506, 515, 522 is interpretable. For instance, let POI #655 be a testsample to try to predict POI #894 (both metros) for user #574.

FIG. 6 shows the corresponding attention plot 600 for one out of theeight GAT layers (preference graph) for newly explored POIs the user hasnever visited before in an embodiment where the GNNs are implemented asGATs.

FIG. 7 show an attention plot 701 for POI-POI attention and an attentionplot 702 for user-user attention in an embodiment where the GNNs areimplemented as GATs. The POIs and users with highest attentioncoefficients are highlighted by boxes. It can be seen that the STP-UGNNgives higher attention coefficients to mostly nearby metros, overdistant malls and the airport. It can also be seen that for user #574the model is attending more to users #594, #687 and #785 than the userhimself/herself. This can be seen as validating the goal of STP-UGNN andsupporting interpretability compared to conventional RNN (RecurrentNeural Network) models.

In summary, according to various embodiments, a method is provided asillustrated in FIG. 8 .

FIG. 8 shows a flow diagram 800 illustrating a method for predicting thedestination location of a vehicle, e.g. predicting the destination of atrip by a user e-hailing a vehicle.

In 801, a local preference graph of a user of the vehicle having nodescorresponding to locations visited before by the user is processed by afirst graph neural network.

In 802, one or more of

-   -   a spatial graph representing information about geographical        proximity of locations,    -   a temporal graph representing information about locations which        have been visited one after another by users and the time        between the visits of the locations and    -   a preference graph representing information about locations        which have been visited one after another by users and the        frequency of visits of the locations are processed by a second        graph neural network.

In 803, the result of the processing by the first graph neural networkand the result of the processing by the second graph neural network arecombined by at least one neural network layer.

In 804, the output of the at least one neural network layer is used asprediction of the destination location.

According to various embodiments, in other words, a local view of theuser (local preference graph), i.e. information depending on a specificuser, is combined with a global view (spatial graph, temporal graph,preference graph), i.e. information independent of users (spatial graph)or depending on all users (temporal graph and preference graph). The setof users may be given by the users of training data used for trainingthe complete model, i.e. the complete neural network including the GNNsand the at least one neural network layer (and possibly further neuralnetwork components as described in the various examples andembodiments).

It should be noted that the method for predicting the destinationlocation can use sequence of past destination locations or past origindestinations or both. Similarly, the training data may include sequenceof past destination locations or past origin destinations or both. Thismeans that the method may use and the training data may include pasttrip information that can consist of origin or destination locations orboth.

According to one embodiment, the destination location is predicted bymeans of a model (e.g. the STP-UGNN described above) which is trained tolearn POI-POI relationships from both local and global views based onspatial, temporal and preference factors by balancing theexplore-exploit trade-offs. The STP-UGNN further includes a third graphneural network to learn (in training) and use (in deployment) user-userrelationships to support the recommendation task.

According to one embodiment, the processing of the spatial graph,temporal graph, and/or preference graph (STP graphs) includes usage of amasked self-attention option of random walks that can leverage the graphstructure to identify and attend higher-order neighbours as compared tojust first-order neighbours in GNN.

Experiments show that new POIs which a user has never visited before butspatially, temporally or preferentially discovered during theexploration phase on the STP graphs can benefit the next POIrecommendation task.

By predicting the destination of the trip, in particular the length ofthe trip can be predicted. This information can be used to select avehicle which is assigned to make the trip, e.g. by checking whether thefuel level or battery level is sufficient for the predicted length ofthe trip.

The prediction of a destination for example allows controlling whichvehicle (e.g. taxi) is assigned to a certain trip. A plurality ofvehicles may in particular be controlled to minimize (or at leastattempt to minimize) empty runs (or the total distance of empty runs)using destination predictions.

The prediction of destinations according to various embodiments may alsobe used for traffic management, e.g. to avoid traffic jams.

Additionally, suggesting a user a destination, e.g. a point-of-interest(POI) where he or she in fact wants to go next increasesuser-friendliness of, e.g. an e-hailing application. For example, theuser is relieved of the burden of searching a map for the destination heor she wants to go or type the name of a destination he or she wants togo.

The method of FIG. 8 is for example carried out by a server computer asillustrated in FIG. 9 .

FIG. 9 shows a server computer 900 according to an embodiment.

The server computer 900 includes a radio interface 1101 (e.g. configuredfor radio communication with the user's smartphone or generally apositioning device to determine the current location of the user, forexample via a mobile radio communication network). The server computer1100 further includes a processing unit 1102 and a memory interface1103. The memory interface 1103 allows the processing unit 1102 toaccess an internal or external memory, e.g. storing information aboutPOIs and users (which can be used as a basis to construct the variousgraphs) or storing the various graphs themselves. The server computer isconfigured to perform the method of FIG. 8 .

The methods described herein may be performed and the various processingor computation units and devices described herein may be implemented byone or more circuits. In an embodiment, a “circuit” may be understood asany kind of a logic implementing entity, which may be hardware,software, firmware, or any combination thereof. Thus, in an embodiment,a “circuit” may be a hard-wired logic circuit or a programmable logiccircuit such as a programmable processor, e.g. a microprocessor. A“circuit” may also be software being implemented or executed by aprocessor, e.g. any kind of computer program, e.g. a computer programusing a virtual machine code. Any other kind of implementation of therespective functions which are described herein may also be understoodas a “circuit” in accordance with an alternative embodiment.

While the disclosure has been particularly shown and described withreference to specific embodiments, it should be understood by thoseskilled in the art that various changes in form and detail may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims. The scope of the invention is thusindicated by the appended claims and all changes which come within themeaning and range of equivalency of the claims are therefore intended tobe embraced.

1. A method for predicting the destination location of a vehicle,comprising: processing a local preference graph of a user of the vehiclehaving nodes corresponding to locations visited before by the user by afirst graph neural network; processing one or more of a spatial graphrepresenting information about geographical proximity of locations; atemporal graph representing information about locations which have beenvisited one after another by users and the time between the visits ofthe locations; and a preference graph representing information aboutlocations which have been visited one after another and the frequency ofvisits of the locations; by a second graph neural network; combining theresult of the processing by the first graph neural network and theresult of the processing by the second graph neural network by at leastone neural network layer; and using the output of the at least oneneural network layer as prediction of the destination location.
 2. Themethod of claim 2, further comprising processing a user-user graphrepresenting information about similarity of users in terms of thelocations they have visited by a third graph neural network andcombining the results of the processing by the third graph neuralnetwork with the result of the processing of the first graph neuralnetwork and the result of the processing of the second graph neuralnetwork by the at least one neural network layer.
 3. The method of claim1, wherein the spatial graph has nodes corresponding to locations, edgesbetween nodes if the locations corresponding to the nodes aregeographically near each other and edge weights depending on thegeographical proximity of the locations corresponding to the nodesconnected by the edges.
 4. The method of claim 1, wherein the temporalgraph has nodes corresponding to locations, edges between nodes if thelocations corresponding to the nodes have been visited one after anotherbased on the visits' timestamps and edge weights depending on the timebetween the visits of the locations corresponding to the nodes connectedby the edges.
 5. The method of claim 1, wherein the preference graph hasnodes corresponding to locations edges between nodes if the locationscorresponding to the nodes have been visited one after another in allusers' historical sequential visits and edge weights depending on thefrequency with which the locations corresponding to the nodes connectedby the edges have been visited one after another.
 6. The method of claim1, wherein the user-user graph has nodes corresponding to users andedges between nodes if the similarity of the users corresponding to thenodes in terms of the locations they have visited is above apredetermined threshold.
 7. The method of claim 1, wherein processing atleast one of the spatial graph, the temporal graph and the preferencegraph comprises selecting a sub-graph of the respective graph andfeeding the sub-graph to the second graph neural network.
 8. The methodof claim 7, wherein selecting the sub-graph of a graph comprisesselecting nodes of the graph by one or more random walks through thegraph.
 9. The method of claim 8, wherein the one or more random walksdepend on the edge weights of the graph.
 10. The method of claim 8,wherein a multiplicity of random walks are performed on the graph andnodes which have been visited most in the random walks are selected forthe sub-graph.
 11. The method of claim 1, wherein processing at leastone of the spatial graph, the temporal graph and the preference graphcomprises selecting a plurality of sub-graphs of the respective graphand feeding the sub-graphs to different sub-graph neural networks of thesecond graph neural network.
 12. The method of claim 11, whereinselecting a plurality of sub-graphs for a graph comprises selecting atleast one sub-graph having nodes adjacent to one or more nodes of thevehicle's user's historical destination visit sequence and selecting atleast one graph selected by at least one random walk through the graph.13. The method of claim 1, comprising processing at least two of thespatial graph, the temporal graph and the preference graph by feeding,for each of the at least two graphs, a first sub-graph having nodesadjacent to one or more nodes of the vehicle's user's historicaldestination visit sequence to a first set of sub-graph neural networks,and a second sub-graph having nodes selected by one or more random walksto a second set of sub-graph neural networks, mean pooling the result ofthe first set of sub-graph neural networks, mean pooling the results ofthe second set of sub-graph neural networks and combining the results ofthe mean poolings by the one or more neural network layers.
 14. Themethod of claim 1, wherein the at least one neural network layercomprises at least one of a dropout layer and a linear layer.
 15. Themethod of claim 1, comprising processing all of the spatial graph, thetemporal graph and the preference graph by the second graph neuralnetwork.
 16. The method of claim 1, wherein one or more of nodescorresponding to locations have trainable features, nodes correspondingto users have trainable features, the graph neural networks havetrainable weights and the at least one neural network layer hastrainable weights.
 17. The method of claim 1, comprising setting one ormore of the trainable features and the trainable weights by a trainingprocedure using historical trips of the users.
 18. The method of claim1, further comprising selecting a vehicle for making a trip with theuser depending on the predicted destination location from a plurality ofcandidate vehicles.
 19. The method of claim 18, comprising predicting atravel distance from the prediction of the destination location andselecting the vehicle from the plurality of candidate vehicles dependingon the predicted travel distance.
 20. The method of claim 19, comprisingselecting the vehicle from the plurality of candidate vehicles such thatthe selected vehicle has sufficient fuel or battery to travel thepredicted travel distance.
 21. A server computer comprising a radiointerface, a memory interface and a processing unit configured toperform the method of claim
 1. 22. (canceled)
 23. A computer-readablemedium comprising program instructions, which, when executed by one ormore processors, cause the one or more processors to perform a methodfor predicting the destination location of a vehicle, the methodcomprising: processing a local preference graph of a user of the vehiclehaving nodes corresponding to locations visited before by the user by afirst graph neural network; processing one or more of a spatial graphrepresenting information about geographical proximity of locations; atemporal graph representing information about locations which have beenvisited one after another by users and the time between the visits ofthe locations; and a preference graph representing information aboutlocations which have been visited one after another and the frequency ofvisits of the locations; by a second graph neural network; combining theresult of the processing by the first graph neural network and theresult of the processing by the second graph neural network by at leastone neural network layer; and using the output of the at least oneneural network layer as prediction of the destination location.